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CONVERGENCE OF MARKOV PROCESSES NEAR 
SADDLE FIXED POINTS 

By Amanda G. Turner 

University of Cambridge 

We consider sequences {X^)t>o of Markov processes in two di- 
mensions whose fluid limit is a stable solution of an ordinary differen- 
tial equation of the form xt = b{xt), where b{x) = ^"^x + r{x) for 
some A,/i > and t{x) = 0{\x\'^). Here the processes are indexed so 
that the variance of the fluctuations of is inversely proportional 
to A'^. The simplest example arises from the OK Corral gunfight model 
which was formulated by Williams and Mcllroy [Bull. London Math. 
Soc. 30 (1998) 166-170] and studied by Kingman [Bull. London Math. 
Soc. 31 (1999) 601-606]. These processes exhibit their most interest- 
ing behavior at times of order log A*' so it is necessary to establish 
a fluid limit that is valid for large times. We find that this limit is 
inherently random and obtain its distribution. Using this, it is pos- 
sible to derive scaling limits for the points where these processes hit 
straight lines through the origin, and the minimum distance from the 
origin that they can attain. The power of A'^ that gives the appropri- 
ate scaling is surprising. For example if T is the time that first 
hits one of the lines y = x or y = —x, then 

^A./(2{A + M))|j^Ar| ^ |^|h/(A + m) 

for some zero mean Gaussian random variable Z. 

1. Introduction. The fluid limit theorem is a powerful result which shows 
that, under certain conditions, sequences of Markov processes converge to 
solutions of ordinary differential equations. We are interested in situations 
where the differential equation can be written in the form 

(1) Xt = Bxt + T{xt), 

for some matrix B, where t{x) = 0{\x\'^) is twice continuously differ entiable. 
These differential equations have been studied extensively in the dynamical 
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Fig. 1. The phase portrait of an ordinary differential equation having a saddle fixed point 
at the origin. 

systems literature, with the aim of finding precise relationships between 
solutions of these differential equations and solutions of the corresponding 
linear differential equations 

(2) yt = Byt. 

We restrict ourselves to the two dimensional case where the origin is 
a saddle fixed point of the system, that is, B has eigenvalues A, —fi, with 
A, ^ > 0. The phase portrait of (1) in the neighborhood of the origin is shown 
in Figure 1. 

In particular, there exists some xq ^ such that (ptixo) — > as t — > oo, 
where (p is the flow associated with the ordinary differential equation (1). 
The set of such xq is the stable manifold. There also exists some x^o such 
that (^^^(xoo) — > as t — > oo. The set of such x^o is the unstable manifold. 
The saddle case is interesting in this setting as it is the only case in two 
dimensions where there is both a stable and an unstable manifold. 

Fix an xq in the stable manifold and consider sequences of Markov pro- 
cesses with initial condition Xq = xq, where the processes are indexed so 
that the variance of the fluctuations of is inversely proportional to N. 
The fluid limit theorem tells us that for fixed values of t, Xj^ — > <j)t{xQ) as 
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(f)t{xo) for large values oft. 



N ^ oo. However, if we allow the value of t to grow with N as N ^ oo, 
we shall see that deviates from the stable solution to a limit which is 
inherently random, before converging to an unstable solution (see Figure 2). 

More precisely, we observe three different types of behavior depending on 
the time scale: 

(A) On compact time intervals, converges to the stable solution of 
(1), the fluctuations around this limit being of order N~^/'^ . 

(B) There exists some xq ^ 0, depending only on xq, and a Gaussian 
random variable Z^o such that if t lies in the interval [i?, ^ log N — R\, then 

= xoe~^\ei + ei) + N~^/^Z^e^\e2 + £2) 

for some £i{t,N) — > uniformly in t in probability as R,N ^ 00, where 
{61,62} is the standard basis for M^. In other words, X^^ can be approxi- 
mated by the solution to the linear ordinary differential equation (2) starting 
from the random point )• 

(C) On time intervals of a fixed length around ^logA^, X^^ converges 
to the unstable solution of (1). 
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The most interesting behavior occurs on time intervals of fixed lengths 
around 2(a+/j) ^"S-^' these values of t the two terms xoe~^^ and 

N~'^/'^ ZooS^^ are of the same order. By considering 

we show in Section 7 that it is at these times that crosses all the straight 
lines passing through 0, and also that \X^\ attains its minimum value when 
t is in this range. The distance from the origin of for these values of t is 
of order A^~/^/(2(A+Ai)) ^ which gives us surprising scaling limits for the points 
at which X^ intersects various straight lines, and for inf 

In order to study the Markov processes at times of order logA^, it is 
necessary to establish a strong form of the fluid limit theorem that is valid 
for large times. The key idea is to show that for N and to sufficiently large, 
the process {X^)t>tQ is close to {(/)t-tQ{Xl^))t>to- This is done in Section 2 
in the case when (1) is linear and X^ is a pure jump Markov process, in 
Section 5 for general pure jump Markov processes, and in Section 6 for 
continuous diffusion processes. In Sections 3 and 4 we look at the process 
{(t>t-to i^i^))t>to foi' large values of and to, which then enables us to obtain 
scaling limits for the process X[^ . The same idea can be used to obtain fluid 
limit theorems for arbitrary matrices B, for example, with eigenvalues having 
the same sign, or in higher dimensions, however an analysis of the solutions 
of the underlying differential equation is required, which we do not go into 
here. 

The simplest example of this type of behavior arises from the OK Corral 
gunfight model which was formulated by Williams and Mcllroy [7] and stud- 
ied by Kingman [5] and Kingman and Volkov [6]. Two lines of gunmen face 
each other, there initially being N on each side. Each gunman fires lethal 
gunshots at times of a Poisson process with rate 1 until either there is no 
one left on the other side or he is killed. The process terminates when all 
the gunmen on one side are dead. It is shown by Kingman that if is the 
number of survivors when the process terminates, then 

where Z ~ iV(0, i). It is the occurrence of the unexpected power of N that 
interested the above authors in the problem. By using our scaling limits we 
rederive this result in Section 2.1 and show that it is a special case of a much 
more general phenomenon, and that in fact by a suitable choice of B, every 
number in the interval (^, 1) may be obtained as a power of in this way. 
An application of the nonlinear case to a model of two competing species is 
given in Section 7. 
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2. The linear case. In this section we restrict ourselves to sequences of 
Markov processes in the special case where equation (1) is linear. We begin 
by describing the conditions under which a limit theorem exists for large 
times and then establish the exact limit by means of an appropriate martin- 
gale inequality. In Section 2.1 this result is used to derive scaling limits for 
the points where these processes hit straight lines through the origin and we 
use this to obtain a solution to the OK Corral problem. 

The fluid limit theorem that we state below is widely known and has been 
the object of many works. We use the formulation found in [2]. 

Let {Xl^)t>o be a sequence of pure jump Markov processes, starting from 
xq and taking values in some subsets of M?, with Levy kernels K^{x, dy) . 
Let S be an open subset of with xq S 5, and set = nS. For x G 
and 6 € (M^)*, define the Laplace transform corresponding to Levy kernel 
K^{x,dy) by 

m^(^x,e)= f e^^'y'^K^{x,dy). 



We assume that there is a limit kernel K{x,dy) defined for x E S" with 
corresponding Laplace transform m{x,9) with the following properties: 

(a) There exists a constant ?7o > such that m{x,9) is uniformly bounded 
for all X G (S and \9\ < rjQ. 

(b) As TV^cx), 



sup sup 

xG5^ \e\<rio 



m^{x,N9) 

— mix, 9) — > 0. 



N 



Set b{x) = m'{x,0) where ' denotes differentiation in 9. Suppose that b is 
Lipschitz on S so that b has an extension to a Lipschitz vector field 6 on . 
Then there is a unique solution {xt)t>o to the ordinary differential equation 
xt = b{xt) starting from xq. Suppose that 5 contains a neighborhood of the 
path {xt)t>o- By stopping at the first time it leaves S, if necessary, we 
may assume that remains in S for all t > 0. Under these assumptions, 
for all to>0 and 6>0, 

limsupiV-MogP( sup|Xf -xtl >(5) < 0. 

N^oo \t<to / 

Suppose additionally that: 
(c) 6 is on S and 

sup N^/'^\b^{x)-b{x)\^0, 
where 6^(x) =m^'(x,0). 
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(d) a, defined by a{x) =m"{x,0), is Lipschitz on S. 

It follows from the above that for any rj <r]o there exists a constant A such 
that 

(3) sup sup N\m^"{x, NO) \ < A. 
xGS'^ \e\<ri 

Let 7^ = N^/^{Xl^ - xt). Then for any t > 0, 7^ ^ 74 as iV ^ 00, where 
(7<)t>o is the unique solution to the linear stochastic differential equation 

(4) djt = (7ixt)dWt + Vb{xt)-ftdt 

starting from 0, W a Brownian motion in M?, and o" € (g) (M?)* satisfying 
a{x)a{x)* = a{x). The distribution of (7t)t>o does not depend on the choice 
of a. 

We are interested in the case where b{x) = Bx for some matrix B = 

Let 04 (x) be the solution to the ordinary differential equation 

(5) (t)t{x) =h{(l)t{x)), (t)Q{x)=x. 

In the linear case we can solve (5) explicitly to get (l)t{x) = e^*x. We 
concentrate on processes where the initial condition is chosen to be xq = 
(xo,i,0) with xq^i 7^ 0, so that xt = (ptixo) — > as t ^ 00. We shall show 
that for sufficiently large values of N and to, is in some sense close to 

Introduce random measures and on (0, 00) x M^, given by 
u^{dt,dy)=K^{X^_,dy)dt, 

N _ vN vN 



where denotes the unit mass at (t, y) and AX^ = X^ — X^ 



Let f{t,x) = e-^\x - (pt-taiX^)) , for t > to- By Ito's formula, 

f{t, ) = /(to, O + Mf - M^f + £ + K^f) is, Xf„) ds, 



where 



^ = -Be-^^x 
dt 



K^f{s,x)= / {f{s,x + y)-f{s,x))K^{x,dy) 
= / e~'''yK^{x,dy) 
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and 



M, 



B,N 



[ (/(s,Xf_ + y)- /(s,Xf_))(/x^ - u^){ds,dy) 

J(0,i]xR2 



e-%(/.^-i.^)(ds,dy). 



/(0,i]xIR2 

So if t > to 5 then 

-BtfvN 



(6) 



B,N T>rB,N 



J to 



Lemma 2.1. There exists some constant C such that 



t>to 



Proof. By the product rule, 

= [\b - A/)e(^-^^)^(Mf - ^) ds 

J to 

+ f I e-^^y{^l''-u''){dy,ds) 



and hence 



E(supe-^*|e^*(Mf'^-M,^'^)| 



\t>to 



< Ef sup [\\ + //)e-(^+'^)^|(Mf - M^'^)i\ds 

\t>tn J to 



+ e( sup 

\t>to 



e-^-'yif,^-u^)idy,ds) 



to JR^ 



Jto 



+ E ( sup 
\t>to 



to Jm.^ 



2\ 1/2 



Since 



E 



e ^^y\iJ^{ds,dy) < oo 
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for all t>0, the process 



is a martingale, and hence, by Doob's inequality 



E( 



sup 

\t>to 



to JR'^ 



<4supE 

t>to 



to JR^ 



Now 



E((Mf'^-M,^'^)?)=E 



to JM2 



<E f\^>''\m^"{X^_,0)\ds 
Jto 



< 

where A is defined in (3). Similarly 

rt 



2fiN ' 



E 



e~^'y{fi''-u''){dy,ds) 



to JK2 



< 



~2Xtoj^ 

2XN 



Hence 



e( 



sup e 

\t>to 



e^*(Mf'^-M,r)l) 

Jtn 



ds + e 



-\to 



[xnJ 



A(2/i)i/2 
Theorem 2.2. Foralle>0, 

lim limsuppf supe"^*|Xf -(/>j_jo(Xj^)| > = 0. 



to— ►oo ^ 



\t>to 



□ 



Proof. Let A^'o be sufficiently large that sup^>jVo 
and set 



n 



N,to 



(sup|e"^*e^*(M,^ 

[t>to 



to 'I — 2 
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By (6), on the set i^N,to with N > Nq, 

t>to t>to 

ft 



t>tn Jtn 



t>to J to 

Hence 

limsuppf supe-^*|Xf - </>t-to (^i^) I > N-^'h\ < nmsupF(0^,,J 

Af-»oo \t>to / N^oo 

< 

£ 

as to — ^ oo, where the second inequahty follows by Markov's inequality and 
Lemma 2.1. □ 

Let Zoo ~ iV(0, o-^), where 

Theorem 2.3. T/ie following converge in probability as — > oo. 
(i) 



for any sequence ^ oo with e^^'^^^^'^ = 0{N^^'^); 



sup |e^*Xfi -xciHO 

t<tjV 



sup 7Vl/2g-At|^7V| 

t>tN 

for any sequence t]\f with e'^^+tA^N = uj{^N^I'^'); 
(iii) 

sup iVV2|e-Ati^JV^^ _ e-^*^X/^_2l 

for any sequence ^at — > oo. 
Furthermore, ifa^^Q, then 

as t,N ^ oo. 
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Remark 2.4. Given any sequence of times tAr^oo as iV— >cx3, by the 
Skorohod representation theorem, it is possible to choose a sample space in 
which = N^^'^e~^^^ X[^^2 ~^ ^oo almost surely as ^ oo. In this case 
the above result can be expressed as 

(7) = xo,ie-'^*(ei + ei) + N~'/^Z^e^\e2 + 82) 

where Ei = Si{N, t) 0, uniformly in t, in probability as ^ 00. 



Proof of Theorem 2.3. For any fixed to, s'^Pt<to \*^'^^^t!i ~ ^0,1 1 
as an immediate consequence of the fluid limit theorem. For (i) it is therefore 
sufficient to show that for any e > 0, limjo^oo limsup^v^oQ P(sup^|j<j^ je'^^X/^ — 
xo,i| > e) = 0. Nowif t > to, 0i-io W) = e^(*-*o)X,^ = e^('-''^Hx~, + N~y\^J. 
Since xo = xo.iei, e^(*~*°)xfp = e~^^xo. Hence 

and so 

and 

(8) Xf, = X-^/^e^*(e-^Sir,2 + ivV2e-A*(xf - «)),). 

Let N ^ 00 and then to ~^ 00. Statements (i)-(iii) follow by Theorem 2.2 
and the fact that 7/^ =^ 7to , ^ Gaussian random variable. 
For the last part, note that by (4), 

J 

as N ^ 00. Since 

roc roc 

/ \e-^'e*2a{xs)\'^ds< / e-^^'\a{xs)\ds 
Jo Jo 

A 

where e* is the transpose of e^, e~'^^°"fto,2 -^oo almost surely as to — > 00, 
for 

Z^=(^J\~^'aixt)dWt^^r^N{0,al). 
The result follows by (8) and Theorem 2.2. □ 
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2.1. Applications. Applications will be dealt with more fully in Section 7. 
However, we illustrate here how the above result can be used to study the 



first time that Xj- hits Ig or l-o, the straight lines passing through the origin 
at angles 9 and —9, where ^ € (0, |), as — > oo. As A/^ is not continuous, 
we define the time that A/^ first intersects one of the lines l±g as 



inil i > : 



< I tan 01 and 



vN 



vN 



> tan t 



Let 



and 



1 



2{X + fi) 



logN 



log 



xo 1 tan 6* 



Theorem 2.5. Under the assumptions of Theorem 2.3 

(9) t;' 

and 

(10) A^/(2(A+/.))|^7y^| ^ 

e 

as N ^ OO. 



Proof. For simplicity, we work in a sample space in which 
almost surely. Define as in Remark 2.4. By observing that 

xo,ie-^*ei + A-i/2^ooe^*e2 

first intersects one of the lines l±0 at time t = t]\f + cq, given any e > 0, 

P(7/ <tN + cg-e) 

•( 



< 



sup 

\t<tM+Cg~e 



sup 



> tan ( 



xo,ie-''*ei,2 + N-^/^Z^e^'{l + £2,2) 



xo,ie-^*(l + ei,i) + A^-i/2Zooe^*e2,i 



> tan ( 







as A^- 



00. Similarly, 
HTg^ >tN + ce+e)< 
The result follows immediately. □ 



inf 



vN 



< tan ( 



0. 
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Remark 2.6. The sign of Z^o determines whether hits Iq or I^q at 
time . Since Z^a is a Gaussian random variable with mean 0, each event 
occurs with probabihty ^. 

Example 2.7 (The OK Corral problem). The OK Corral process is a 
1? valued process {U^^ ,Vj-^) used to model the famous gunfight where C//^ 
and Vj^ are the number of gunmen on each side and Uq^ = Vq^ = N. Each 
gunman fires lethal gunshots at times of a Poisson process with rate 1 until 
either there is no one left on the other side or he is killed. The transition 
rates are 

, X ({u — l,v), at rate f, 
' \{u,v — l), at rate li 

until uv = 0. 

The process terminates when all the gunmen on one side are dead. We are 
interested in the number of gunmen surviving when the process terminates, 
for large values of A^. 

This model was formulated by Williams and Mcllroy [7] and later studied 
by Kingman [5] and subsequently Kingman and Volkov [6]. 

Let = {U^ , V^)/N . This gives a sequence of pure jump Markov pro- 
cesses, starting from = (1, 1), with Levy kernels 

If we let 

K{x, dy) = 3;25(-i,o) + a^i V-i)' 

then 

m{x,9) =X2e-^' +xie-^^ = -, 

and 

So, under a rotation by j, the conditions required for Theorem 2.5 are 
satisfied, with X = fi = l. In the original coordinates, the process terminates 
when X^^ hits the x ov y axes. Under the rotation, this corresponds to hitting 
l±2L. Hence, if the OK Corral process terminates at time Tjy and there are 

4 

survivors, then T/v = and = N\X^n |, and so 

' 7r/4 

Ttv - i log TV ^ i log 2 - i log I Zoo I 
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and 

where Z^o ~ -/V(0, |). The hmiting distribution of N~^/^S^ is the one ob- 
tained by Kingman in [5]. 

Remark 2.8. It is remarked by Kingman [5] that it is the occurrence 
of the surprising power of A'^ that makes the OK Corral process of interest. 
Theorem 2.5 shows that this is a special case of a more general phenomenon, 
and in fact, by a suitable choice of ^, every number in the interval (|,1) 
may be obtained as a power of N in this way. 

3. Linearization of the limit process. We now turn to the general case 
where b{x) = Bx + t{x) for B = {"^^ /i, A > 0, and t -.M? ^ M?, twice 
continuously differentiable, with t(0) = Vr(0) = 0. Let 4>t{x) be the solution 
to the ordinary differential equation 

(11) (t)t{x) =h{(t)t{x)), (Pq(x)=x. 

This section consists of a technical calculation which expresses (j)t{x) in a 
linear form. 

We are interested in the behavior of solutions starting near the stable 
manifold. Lemma 3.2 proves the existence of the stable manifold and es- 
tablishes the limiting behavior of a stable solution. First order behavior is 
investigated in Lemma 3.3, and these results are then used in Theorem 3.4 
to express solutions near the stable manifold in the required linear form. 
Theorem 3.5 shows that over large time periods, solutions starting near the 
stable manifold approach the unstable manifold. 

Throughout this section we use the following classical planar linearization 
theorem due to Hartman [4]. 

Theorem 3.1. There exists a dijfeomorphism h:U ^ V = h{U), 
defined on an open neighborhood U of the origin, with uniformly Holder 
continuous partial derivatives and having the form h{x) = x + o{x) such that 

h{Ux)) = e'''h{x) 

for all {t,x) with (j)t{x) € U. 

Pick < 6 <1 sufficiently small that the ball of radius S centered at the 
origin is contained in UCiV. Since h~^{x) =x + o{x), and V/i(x) =/ + o(l) 
we can further ensure that S is sufficiently small that 

sup {\h{x)/x\ V \h-\x)/x\)<2 

0<\x\<S 
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and 



sup (|V/i(x) - /I V |V/i~^(x) - /|) < 1/2. 

\x\<S 



Lemma 3.2. There exists an xq with < |xo| < S/8 such that (pti^o) 
as t oo. Furthermore, for any such xq, there exists some xq with < |xo| < 
6/4 such that 



as i — > oo, and 



for all t > 0. 



^'t(2;o)| <2|xo|e-^*<<5e-^V2 



Proof. Pick some xq GM with 0< I^o I <V16 and define xo = /i ^(xo,0). 
Then 

5 



0<|xo|< sup \h "'"(x)/2;||xo| < 

0<\x\<S 



and 



h- 



-i ( e ^^^xq 








as t — > oo. 

Conversely, given xq satisfying the above conditions, define xq = /i(xo)i. 
Note that because of the form of h{x), xq has the same sign as xo,i. Since 
e^*/i(xo) = h{4>t{xQ)) ^ as t ^ oo, /i(xo)2 = 0, and so 



Also 

as i — > oo, and 

\4>t{xQ)\ 

for all t > 0. □ 



< |xo| = \h{xQ)\ < 2|xo| < 5/4:. 



^^,tlle-^'x^ 







-lie '^^xq 



Xo 









- I ui 2 



Lemma 3.3. (i) There exists some Do € (M.'^)* \ {0}, where = (0 0), 
such that 



e-^'Vct>t{xo) 





^0 



as t ^ oo. 
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(ii) // |x| < 5 and |</'f(x)| < 6/2, then |V(/)j(x)| < 4e^*. 

(iii) // |a:| + |?/| < (5 and supo<9<i + 6y) \ < 6/2, then there exist con- 
stants K (^M. and < a < 1 such that 

Proof, (i) Let Dq = Vh2{xo) G (M^)* \ {0}. Then 





JV/i(xo) 





^0 

as i — > oo. 

(h) If \(j)t{x)\ < 6/2, then |e^*/i(x)| = \h{4>t(x))\ < 6 and so 

= \Vh~\e^'h{x))e^'Vh{x)\ 
< sup |V/i~^(y)| sup |V/i(y)|e^* 

\y\<S \y\<5 

<4e^*. 

(iii) Since h and h^^ have uniformly Holder continuous partial deriva- 
tives, there exists some Ko € M and < a < 1 such that 

\Vh{w)-Vh{z)\<Ko\w-z\'' 

and 

\Vh-\w) - V/i~^(z)| < Ko\w - zl". 

Therefore 

|V(/)j(x + y) - V(^t(x)| 

= \Vh-\e^'h{x + y))e^'Vh{x + y)- Vh-\e^'h{x))e^'Vh{x)\ 
< \Vh~\e^^h{x + y))e^\Vh{x + y)-Vh{x))\ 

+ \{Vh-\e^'h{x + y)) - Vh-\e^'h{x)))e^'Vh{x)\ 
< 2e^*Jfo|yr + 2e^'Ko\e^\h{x + y) - h{x))\'^ 

<8iroe^*(^+")|?/r. □ 
Suppose z gM?, with < |z| < 1, and Xz = xo + z. 
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Theorem 3.4. Fix C and consider the limit z —fO with Itt— I < C, where 
Dq is defined in Lemma 3.3. There exist Wi, i=l,2 (not necessarily unique) 
with Wi{t, z) ^ uniformly in t G [R, — log |z| — i?] as z — > and oo 
such that 

Mxz)=xoe~''\ei+wi) + Doze^\e2 + W2). 
Proof. Suppose \^\ < C and R> ^log g_4|^p| - If \x - xo\ < \z\ and 



0<t<( inf infis >0: > 

\\x~xo\<\z\ [ 



A (4 log N-/; I, 



then 



\Mx)\ < \Mxo)\ + \Mx)-Mxo)\ 

< 2|xo|e-'^* + \V4>t{xo + 0'{x - xq))\\x - xq\ 
<2|xo|e-'^* + 4|z|e^* 
<2|xo|+4e-^^ 
5 

where 9' G (0, 1). Hence |</>t(a;)| < 5/2 for all — xo| < l^l and t < — ^ log \z\ — 
R. Now 

4>t{xz) = M^o) + V(pt{xo)z + (Vcptixo + 0z) - V(l)t{xo))z 
for some 6 G (0, 1) and so, defining 

wi{t,z) = Xq ^(e^Vt(a;o) - ^oei) 

and 

W2{t, z) = {Doz)-\e-^'VMxo)z - Doze2 + e-^^V^xo + Oz) - V^t{xo))z), 

Mxz) = xoe-''\ei + wi) + Doze^\e2 + W2). 
Then \wi \ uniformly in t > i? as i? ^ 00 by Theorem 3.2, and 



\z 

\W2\ < 



e"^'WMxo) 



Doz\ 

e~^'V(Pt{xo) 





^0 



+ Ke^'^^z 





Do 



+ Ke~^'^^ 



<C{ 

uniformly in t G [R, — ^ log — i?] as i? ^ 00 and 2; ^ 0, by Lemma 3.3. □ 
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Since (j)'j~^{x) satisfies (11) with b replaced by —b, we may apply Lemma 3.2 
and Lemma 3.3 to deduce the existence of Xqo with < |xoo| < 5/8 such 
that e^^(p'i~^{xoo) ) for some Xoo G M as t ^ oo, and D^o such that 

e-^*V(^("^(xoo) (^0°°) as t ^ oo. Suppose that as z ^ 0, the sign of Doz is 
eventually constant and nonzero. As x^o has the same sign as Xoo,2 (see the 
proof of Lemma 3.2), we may choose Xoo such that > 0. 

There exists some too ^ such that (j)t{xo) does not intersect the line 
+ for any t > too • Let 

Sz = inf{t > too ■ 4>t{xz) = + '''D*o^ for some r € M}. 

Theorem 3.5. Fix C > and consider the limit z ^ with \-f4—\ < C . 
Then 

- - log — > 

A DnZ 



and 



as z — > 0. 



oo 



TT^) {4>sAXz)-Xoo)^Xo- 2 



Proof. We shall prove this theorem in the case where for z sufficiently 
small -Do -2, xq > 0- The other cases are similar. 

Since = ^34^ ^ ^ = as t ^ oo, there exists some T > such 

that \^rf^\ < 1 for all t > T. Let 

tz = ini{t >T:\cl)t{xz)i\ = If^-f (x^)2|}. 

By expressing (j)t{xz) in the form derived in Theorem 3.4, we may use a 
similar argument to that in Theorem 2.5 to show 

log — > 



A + Dqz 

as 2: — > 0. Let f:B{0,l) ^ R be defined by f{z) = (t>tz{xz)i- Again as in 
Theorem 2.5, 



{Doz)-''^^^+''>f{z 



as z ^ 0. 

Define 5 : M+ ^ M by 
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where t'y is defined in the same way as except for instead of cj). 
[The scaling factor of |DooP is chosen so that Doo{yj^^) = y-] Note that 

By a similar argument to above, y^^^^'^^^^ g{y) xi^^^~^^^ as y — > 0. But 
then 

"I y-v(A+/.)g(y) ; 3[y)) I 

^0 

as z — > 0, where y = g~^{f{z)) ^ as z ^0. So 

\fi/x ^f^\^^/■^ . D* D* 



.DqzJ \DqzJ \DooV \Docr 

Also, since t'y = Sz — tz, and t'y — log ^ ^ as y — > 0, 

-iz) - T^^°§77; — >0, 

A + /i (L>o2:/xoo)''/-^rEo 



that is, 



1 . X, 



^z-T- log ^0. p 

4. Convergence of the fluctuations. Now suppose X^^ is a pure jump 
Markov process satisfying all the conditions in Section 2, except with b{x) = 
Bx + t{x), B and r defined as in Section 3. In this section we express 
(j^t-to i-^to ) ^ linear form for large values of and to • 

Recall from Section 2 that 7^ = N^^'^{X^ — xt) and 7/^ =^ 74 for each 
t as A — > 00, where (7t)t>o is the unique solution to the linear stochastic 
differential equation (4). 

Fix some to > 0. Then (f>t-to{X^) = (t>t{4>to i^u^)) S'^d using the same 
notation as in Section 2, there exists some G (0, 1) such that 

^I,\X^,)=^T'M+N-'/'V<l^i^\x,,)^Z 



+ N-^/\v<t>i^\x,, + eN-^i^^Z)-'^KM)% 

xo + N-'/'zl 
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where Z/^ =^ Zt^ = ^ (PT^ ^xta)-tto as iV ^ oo. Now 

D^Zt^ = lim e^e~^*V</.t(xo) / ° V0;i(x,)(j(x,) dW^ 

= lim e^e-^* / " V0j_,(x,)a(x,) dVF, 

i-»oo JO 

and 

POD 

liminf e"^-^* / \e2V(j)t-sixs)(T{xs)\'^ ds 

poo 

<liminfe-2At / |v,^t_,(x,)2p|a(x,)| ds 
<liminfe-2^* H lQ\Ds\\^^^'-'^ Ads 

*^oo Jo 
32y4 

where A is defined in (3) and the modulus of Ds = \\m.t^oo&~^^^ (t>t{xs)2 is 
bounded above by 2, by the same argument used to show existence of Dq in 
Theorem 3.3. Hence, if we define 

roo 

(jI^= I pm. e~'^^^V(t)t-s{xs)2a{xsW<t)t-s{xs)*2ds 
Jo 

e-'^^'D,a{x,)Dl ds, 



then DqZiq — > Z^o almost surely as to oo, where Zoo ~ -^(0, o"^)- 

Choose and x^, with < < 6/2 and 2 < < 2' such that 
— > as t — > 00. Define a random variable on the same sample 
space as Zoo by 

{Xqo ) if Zoo > 0, 
0, if Zoo = 0, 

x~, ifZoo<0, 

and define similarly, except replacing by x^. 

By the Skorohod representation theorem, we may assume we are working 
in a sample space in which Z^ Zt^^ almost surely for all to £ N. Without 
this assumption, analogous results about weak convergence hold, however 
this assumption simplifies the formulation. Let 

(12) SN,to = inf {s > too : ^s~to (Xt^, ) = ^00 + rDl, for some r € M} 
and 

(13) 5Ar = ^logiV+^log-^°° 



2A ° A ° Z 



00 
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where we interpret § = 1- 

Theorem 4.1. Suppose aoo 7^ 0. 

(i) As N ^ oo and then to — > oo, 

e^*|</.t_t„«)-0t(xo)HO 

uniformly in t on compacts in probability. 

(ii) If R <t < ^\ogN — R, then there exist e'^{N,tQ,t) 0, uniformly 
in t in probability as R, N ^ oo and then to ^ oo such that 

(l^t-tMZ ) = Soe-'^*(ei + e'l) + N-^I^Z^e^\e2 + 4)- 

(iii) ^ oo and then to oo, S^^to — Sn ^ in probability. Fur- 
thermore, if t = S^fy — s for some s, then 

e^^\^t-to{XO-cl>;\X^)\^0 

uniformly in s on compacts, in probability as N ^ oo and then to — > oo. 

Proof, (i) By Theorem 3.3, for some 9 G (0, 1) 



^0 

uniformly in t on compacts in probabihty. 

(ii) Apply Theorem 3.4 with z = iV'^/^^Af and use the fact that -Do^t^ ^ 
Zoo almost surely as ^ oo and then to ^ oo. A potential problem arises 
when Zqo is close to 0, however as it is a Gaussian random variable, the 
probability of this occurring can be made arbitrarily small. 

(iii) The first result follows from Theorem 3.5 by a similar argument to 
(ii). For the second result apply a similar argument to the proof of (i) to 

□ 

5. A fluid limit for jump Markov processes. We now show that for large 
values of N and t, is in some sense close to (^t-toi^tl^) ^ and 
combine this with results from Section 3 to obtain results analogous to those 
in the linear case in Section 2. 

Let /(t,x) = e-^*(x - (t)t-to{X?o)) ■ Ito's formula, 

/(t, Xf) = /(O, X^) + + + Kf^ (s, X^_)ds, 
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where 



df 



and 



Mf'^= / if{s,X^^ + y)- f{s,X^_))if,^ -u^){ds,dy) 



(0,t]xM2 

e-%(/i^-i/^)(ds,(i2/). 



/(0,i]xIR2 

So if t > to) then 

-BttvN 1 fvN\ 



(14) 



\x- - 4>t-tMZ)) 

+ re-^^(6^(Xf_)-6(Xf_))d« 

J to 
to 

Since r G C^, Vr is Lipschitz continuous on the unit disc with Lipschitz 
constant denoted by Kq. In addition to the restrictions on 5 from Section 3, 
suppose 6 < 9j^,,^^+^) • 

Theorem 5.1. For all e >0, 

hm hmsuppf sup e-^*|Xf - > eN-^/A = 0. 

t0->0 AT^^oo \tQ<t<SN,to ' 

Proof. Let 

i?;v,to = inf {t > to : e-^* |Xf - 4>t_t, (X^ ) \ > N-^^^e} A SN,to ■ 

We shall show that -Rat,*,) = S]\f,to by bounding the terms on the right-hand 
side of (14). 

Fix c > 0. Since increasing e decreases the above probability, we may 
assume < e < % A Suppose C > 4 and pick R > j^log{^^^^j^) . 
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|supe-^*|e^*(Mf'^-M,f^)|<iV-V2^j, 

it>to o J 

( sup e'^Vt-to(<)-'^i(^o)|<^j 

n( sup |e;(iV,io,i)| V|e'2(Af,to,t)| <lj 
VR<t<SN,tQ-~R J 

n{ sup e^(^-^'o-)|<^,_,jX,^)-0-i ^^(X^)|<n, 
where e'^^ and £'2 are defined in Theorem 4.1, and 

Let A^o be sufficiently large that sup7v>Aro iV^^^H^^ - b\\ < Xe/3. 

On the set Qj^^^ n ^N,to,R ^ ^N,to,c ^ < l^ool < C} with N > Nq, if 
to <t < R, then 

|<^,„,„«)|<<5e-^*, 

ii R<t < Sj\f,tQ — R, then 

\c^t-to{XO\ < |So|e-^*(l + 141) + iV"i/2|Zoo|e^*(l + |4|) 

- 2 

and if S ]\f,to — Rl£t < ^Nfy > then 
From (14), 

Jto 

+ e-^* /* |e^(*-) I |r(Xf_) - r{^s-to {X^ )) I 
•'to 

<e-^*|e^*(Mf'^-il/Ol + i||6^-6|| 

+ /* e-^^ I Vr (<^,„i„ (X,^ ) + e {Xt -<t>s^t, {XZ )))\\Xt-(t>s^t, (X,^ ) | ds 



Define 
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A 

J to 

for some 9 £ (0, 1). 

Hence, on Q^^^^ n n%to^R n 0^^^ ^ n {C^^ <\Zoo\< C} with > A^o, 

sup e-^*|Xf -<^i_i„«)| 

to<i<-R]v,tQ 

<Ar-i/2£ + Ar-i/2£ 
- 3 3 

rRN 



+ Ko ' {\4>s-toiX[',)\ + \X^_-<^s-toiX[!,)\)N-'/hds 

J to 



+ 



/to 



1/9 /2 /(5(A + u) ee-^'^ 2Ce ^R 



3 ' + ^ + 

< N-'/h. 

Since X/^ is right continuous, this means RN,to = SN,to and so 

Pf sup e-^*|Xf -c/>i„i„(X,^)|>iV-i/2£) 
\to<t<5iv,to / 

< m^ktoT) + ip((^^^,t„,ij)^) + m^NMucT) -n\Zoo\^ {c^\c)). 

Letting N, to, R,C,c ^ oo in that order, and using Lemma 2.1 and Theo- 
rem 4.1 gives 

Um hmsuppf sup e'^^lX^ - > N~^/^e) = 0. 

to^OO ff^^ \t<SN,to J □ 

Remark 5.2. The same idea can be used to obtain convergence results 
for arbitrary matrices B, for example, with eigenvalues having the same sign 
or in higher dimensions. The rate of convergence and the time up to which 
convergence is valid will depend on the eigenvalues of B and bounds on 



Combining the above result with Theorem 4.1 we get the following. 
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Theorem 5.3. (i) For all N £N, 

is bounded uniformly in t on compacts, in probability. (This follows di- 
rectly from the fluid limit theorem and diffusion approximation stated in 
Section 2.) 

(ii) Suppose R<t< j^logN — R. Then provided a^o 7^ 0, for i = 1,2 
there exist £i{N, — > uniformly in t in probability as R, N ^ oo such that 

= xoe-'^*(ei + ei) + N-^'^ Zooe^\e2 + £2), 

[cf. (7)]. 

(iii) As N ^00 

uniformly on compacts in s>0, in probability. 

Remark 5.4. These results can be reformulated as results about weak 
convergence which are true, independent of the choice of sample space, in a 
manner analogous to Theorem 2.3. In particular, for any sequence tj\[ —kx) 
as N ^ 00, = N^/'^e~'^^^ Xj^^ 2 =^ ^00 and working on a space in which 
this sequence converges almost surely is sufficient for Theorem 5.3. 

6. Continuous diffusion Markov processes. Interest in this problem arose 
through looking at the OK Corral problem. It was therefore natural to prove 
results for pure jump Markov processes. However the proof of the analogous 
result in the case of continuous diffusion processes is similar and we give 
it below. The pure jump and continuous cases can be combined to obtain 
results for more general Markov processes. 

Let (Xf )j>o be a sequence of diffusion processes, starting from xq and 
taking values in some open subset S" C M^, that satisfy the stochastic differ- 
ential equations 

dXf = ) dWt + 6^(Xf ) dt 

with ,b^ Lipschitz. 

Suppose that there exist limit functions h{x) = Bx + r(x), with B and r 
as in Section 3 and a, bounded, satisfying 

(a) 



(b) 



snvN^''^\b^{x)-h{x)\^Q. 



sup|A^i/V^(x) -c7(x)H0. 
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It follows that there exists a constant A such that for all N 

(15) \\a^\\<{A/N)^/\ 

Let 7/^ = N^/'^{X^ — xt), where xt is defined as before. It is straight- 
forward, using Gronwall's lemma, to show that 7/^ — > 7t as N 00, where 
i7t)t>o is the unique solution to the linear stochastic diff'erential equation 

(16) djt = a{xt)dWt + Vb{xt)jtdt 
starting from 0, W a Brownian motion. 



Consider for t > f(t,x) = e ^^{x — (pt-toiX^)). By Ito's formula. 



/(t,Xf) = /(to,X,^) + Mf'^-M,f^ + £(|{(.,Xf) + e-^^6^(Xf) 



ds, 



where 



and 



So if f > to, 



(17) 



^ = -i?e-^*x-e"^V(<A*_*o(^*o)) 



Jo 



'^*(Xf-,^,_,,«)) 



f e-^%b^{Xf_)-b{X^_))ds 

e-^^{T{Xt)-T{<l^s^t,{X^^)))ds. 



'to 

H 

^-Bst^r vN \ X 

By comparing this with (14), it is sufficient to prove an analogous result to 
Lemma 2.1, for the conclusion of Theorem 5.3 to hold for diffusion processes. 

Lemma 6.1. There exists some constant C such that 

Efsupe-^*|e^*(Mf'^-M,^'^)|') <CiV-V2e-Ato. 

\t>to J 

Proof. By the product rule, 

J to 

+ f\-^^a^{Xf)dWs 

J to 
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E(supe"^'|e«*(Mf'^-M,f^)|) 

\ t>to / 

< Ef sup /*(A + /i)e"(^+^)^|(Mf - Mtf^)i\ds 

\t>toJto 

+ Efsup f\-^'a^{X^)dWs] 

\t>to J to J 

< r(A + /.)e-(^+'^)^(E(Mf'^ -M,^'^)2)V2rfs 

Jtn 



+ E ( sup 

\t>to 



Jto 



2\ 1/2 



Since 



E f' \\e-^'a^{Xf)fds<oo 
Jo 



for all t > 0, the process 



fj^^e-^'a''{X^)dw)i 



t>o 



is a martingale, and hence, by Doob's inequality 



e( 



sup 

\t>to 



[\-^'a^iX^)dWs 

Jto 



<4supE 

t>to 



f\-^'a^{Xf)dWs 
Jto 



Now 



E((Mf - ^)?) = E r e2^^a^(Xf ds 

J to 



to 

rt A 

<E / e^^''—ds 
~ Jto N 



< 



2fiN ' 



where A is defined in (15). Similarly 



E 



to 



< 



-2\toj^ 

2XN ■ 
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Hence 



V t>t() / 

A(2/i)V2 e . ^ 

Define Uoo , -^oo j -^oo , ^oo as in Section 4 and let 
5Ar = ^logiV+ilog^°° 



2A ° A ° Z, 



oo 



The following analogous theorem to Theorem 5.3 for diffusion processes 
holds. 



Theorem 6.2. (i) For all N £N, 

is bounded uniformly in t on compacts, in probability. 

(ii) Suppose R<t< j^logN — R. Then provided a^o 7^ 0, for i = 1,2 
there exist ei{N, t) — > uniformly in t in probability as R, N ^ 00 such that 

= xoe-f^'iei + ei) + N~^/^Z^e^\e2 + £2), 

[cf (7)]. 

(iii) As N ^00 

uniformly on compacts in s> 0, in probability. 

7. Applications. Throughout this section we work in a sample space in 
which Zoo almost surely so that, in particular, the statement of The- 

orem 5.3 holds. 



7.1. Hitting lines through the origin. As in the linear case. Theorems 5.3 
and 6.2 may be used to study the first time that X[^ hits Ig or Lg, the 
straight lines passing through the origin at angles 6 and —0, where € (0, 
as N ^ 00. We define the time that X^^ first intersects one of the lines 1±q 
as in Section 2 by 



infU>0: 



vN 



< I tan 01 and 



vN 
^t,2 



vN 



> tan i 
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First note that by Lemma 3.2, 

Mxo)2 e^'*(t)t{xo)2 



1 = 



(l>t{xQ)i e/^Vt(a;o)i xo 

as t — > oo. In particular, since tan^ ^ 0, there exists some > such that 
1 I < I tan 01 for all t> sg. To rule out the trivial case where TI^ con- 

I (j>t(xo)i I I I — f 

verges to the first time that (j)t{xo) hits l±e, we shall assume that xq is chosen 
sufficiently close to the origin that sg = 0. 

We prove the following result in the case where is a pure jump pro- 
cess. The proof for continuous diffusion processes is identical, except uses 
Theorem 6.2 in place of Theorem 5.3. 



Theorem 7.1. Under the conditions required for Theorem 5.3 



Tg^ — tN ^ eg 



and 



^^/{2{A+;.))|^7y^| 



I sec 1 1 tan 1 |xo 1^/^^+'') | 



as N ^ oo, where 

tN 



2{X + fi) 



log N and cg 



log 



xo tan 9 



Proof. By the ffuid limit theorem and diffusion approximation, for any 
constant i? > 0, 



P(Te <R) < P( sup 

.t<B. 



vN 



vN 



> tan ( 



sup 

t<R 



<^t(xo)i + iV-i/27fi 



> tan t 







as ^ oo. 

By an identical argument to Theorem 2.5 

F{R<T^ <tN + cg-e)^0 

and 

+ ce + e<Tg^ <Sn-R)^0 
as R, N ^ oo. The result follows immediately. □ 
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Remark 7.2. As in the linear case, the sign of determines whether 
hits Iq or I^q at time . Since Z^o is a Gaussian random variable with 
mean 0, each event occurs with probability ^. Furthermore, provided Xqo is 
chosen sufficiently close to the origin that <^i^{xoo) does not intersect /j-g, 
if Xf hits one of the two lines then the probability of it hitting either line 
again before Spf converges to as — > oo. 

7.2. Minimum distance from the origin. Another application is to inves- 
tigate the minimum distance from the origin that X[^ can attain for large 
values of A^. 

Theorem 7.3. Under the conditions required for Theorem 5.3, 



ArM/{2{A+A.)) inf |xf I 

t<SN \XJ \fl 

as N ^ oo. 



/ nA/(2(A+a.))/x xl/2 



Proof. By the fluid limit theorem and diffusion approximation, for any 
constant i? > 0, 

inf ^M/(2(A+M))|^JV| > .^f iVM/(2(A+M))(|^^(^g)| _ Ar-i/2|^JV|) ^ ^ 

as N ^ oo. 

By Theorem 5.3, 

inf iVM/(2(A+M))|^^| 

R<t<tN~B. 

R<t<tN~R 

— >■ oo 

in probability as R,N ^ oo. 

For each c > there exists some e ^ in probability such that 

inf ArM/(2(A+M))|^7V|> .^f ^^/(2(A+/.))(|^-l(^^)| 

— e) — > oo. 

5jv— c<t<5jv 0<s<c 

Also 

inf ^A./{2(A+,.))|^JV[ 
tN+R<t<l/(2X) log AT-R 

> inf e^(*-*^)jZoo|(l-|e2|)-e^(*^-*)|xo|(l + |ei|) 

tN+R<t<l/{2X) logN-R 
OO 

in probability as R, N ^ oo. 
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Finally if t = t^v + c, then 

+ (e-^*xoei,2 + N~"'^Z^e^\l + £2,2))')'/' 

in probability uniformly in c on compact intervals. The right-hand side is 
minimized when 

- ' .log-^^° 



Therefore 



2(A + /z) ^ZIX- 



m/(a+m) 

00 I 



as ^ 00. □ 



{u,v) 



Example 7.4. Let (C//^, 1^^) be a valued process modelling the sizes 
of two populations of the same species with Uq^ = Vq^ = N. The environment 
that they occupy is assumed to be closed with the initial population density 
independent of N. Each individual reproduces at rate 1. Additionally, the 
individuals are in competition with each other, a death occurring due to 
competition over resources at rate a and due to aggression between the 
populations at rate /3. Hence the transition rates are 

{u + l,v), at rate u, 

(n — l,v), at rate au{u + v — l)/N + Puv/N, 

{u, V + I), at rate v, 

{u, V — I), at rate av{u + v — l)/N + fiuv/N . 

Let = {U^ , V^)/N . This gives a sequence of pure jump Markov pro- 
cesses, starting from xq = (1, 1), with Levy kernels 

K^{x, dy) = Nxl5^l|N^Q) + N{axi {xi +X2-I/N) + /3j;iX2)(5(_i/7v,o) 

+ iVx2(^(0,l/Ar) +N{aX2{xi+X2 - 1/iV) + /?XiX2)5(o,-l/Ar) • 

If we let 

K{x, dy) = rEi5(i^o) + {o^xl + /3xia;2)5(_i,o) + a;2<^(o,i) + («^2 + /?2;iX2)5(o,-i) 
then for S = (0, 2f and r?o = 1, 

m{x, 6) = xie^^ + {axl + j3xiX2)e~^^ + X2e^^ + {ax\ + PxiX2)e~^^ 
satisfies 

m^{x,N0) 



sup sup 



N 



m{x, 0) 
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as — > oo. Therefore 

.w^»'(.,o,^(-<;:-:(°+«-)). 

The deterministic differential equation 

<j)t{x) = b{(l)t{x)),(j)o{x) = X 

is a special case of the Lotka-Volterra model for two-species competition. 
See [1] for a detailed interpretation of the parameters a and /3. Further 
generalizations are discussed in [3]. 

It is straightforward to check that b{x) is on S and satisfies 



sup N^/^\b^{x)-b{x)\^0 



as N ^0, and that 
a{x) 



xi{l + axi + {a + i3)x2) \ 

X2{1 + ax2 + {a + 13) xi) J 

is Lipschitz on S. 

Now b{x) has a saddle fixed point at ( 2a+0 ' 20+/; ) ^'^'^ symmetry any 
point X on the line xi = X2 satisfies (^t(x) —>■ ( 2a+p ' 2a+/3 ) ^® t ^ 00. So 
under an appropriate translation and rotation, the conditions required for 
Theorem 5.3 are satisfied, with A = 1 and fi = 2a+p ■ [^ote that > since 
a{x) is positive definite on S]. Hence for times t satisfying t <^tN, where t]y = 
4{a+^) 1°S-^) the two populations coexist with the sizes of both being equal. 
However at time + 0(1) the deterministic approximation breaks down 
and one side begins to dominate. The previous results give a quantitative 
description of the behavior of the processes in this region, however we do 
not go into this here. At time t = Sn + s = ^logN + 0(1), (l)-^{Xoo) 
in probability as N ^ 00, where Sjsf is defined in Theorem 5.1 and Xqo is 
defined in Section 4. Now b{x) has stable fixed points at (a~^, 0) and (0, a~^) 
and hence (pJ^{X^) converges to one of these two fixed points as s ^ 00. 
For any e E (0, 1) we say that a population is e-extinct if the proportion of 
the original population that remains is less than e. Thus for arbitrarily small 
e, one of the populations will become e-extinct at time ^ log -|- 0(1). 
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